Integrated consistent and complete “expert” and data mining

1. INTRODUCTION. Integration of knowledge management (KM) and Data Mining (DM) methods can benefit many applications. It permits to combine and mutually verify knowledge obtained from experts and extracted from raw data. Traditional expert systems rely on knowledge " extracted " in the form of If-Then diagnostic rules from experts. Systems based on Machine Learning technique rely on an available database for discovering diagnostic rules. These two sets of rules may contradict each other. An expert may not trust rules, as they may contradict his/her existing rules and experience. Also, an expert may have questionable or incorrect rules while the data/image base may have questionable or incorrect records. Moreover, data mining discovery may take the form different from If-Then rules and these rules may need to be decoded before they are compared to expert rules. For high-risk applications such as financial investment and life-critical medical applications, e.g., for breast cancer diagnosis, benefits of wise integration are especially evident [Kovalerchuk, Vityaev, Ruiz, 2000]. Suppose that a DM method extracted an " excellent " diagnostic rule, which is near 100% correct on the data used for discovering and testing the rule. On the other hand, assume that this rule contradicts the opinion of an experienced expert. Who will risk relying on such rule without extra analysis, e.g., for cancer diagnosis? An expert can argue that the rule was extracted from a non-representative database (DB) even if it is a large one. For instance, the DB contains a huge amount of negative examples (mammograms of benign cases) and just few positive (cancer) cases. Not the full variety of positive cases may be fully represented in the database. Similarly in risky investment, a buy/sell signal generated by a DM method may contradict an opinion of an experienced trader/investor. Integration of DM and KM methods can help to identify such situations beforehand, saving lives, money and bringing other benefits. Next, working on producing a consistent result may reveal a source of contradiction (e.g., a non-representative database or an unmotivated expert's opinion). Finally, this will build a foundation for better-combined results. In this paper knowledge management and data mining techniques are integrated using two methods-one from the KM area and other one from the DM area. The methods and their combination are powerful and unique in some sense. The first method is focused on knowledge acquisition from humans directly via dynamic optimized expert interview. It is …